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the  multi-processor  workstations  have  allowed  us  to  develop  and  test  our  Navier-Stokes  codes 
locally  and  improve  their  performance  before  performing  production  rans  at  the  DOD 
High-Performance  Computing  Centers. 
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DURIP  Grant  No.  F  49620-95-1-0123 


from  AFOSR/NI 


Advanced  Pre-  and  Post-Processing  Eqipment  for  Time-Dependent  Numerical 

Simulations  of  Complex  Flows 

by 

Hermann  F.  Fasel 


Abstract 


With  the  funding  from  DURIP  grant  F  49620-95-1-0123  a  total  of  seven  workstations  from  Silicon 
Graphics  Inc.  and  several  peripherals  and  accessories,  including  a  36  GB  disk  array  have  been  purchased. 
This  equipment  has  been  utiUzed  effectively  to  perform  pre-  and  post-processing  tasks  for  our  DOD 
funded  research  projects.  The  instrumentation  is  very  well  suited  to  perform  our  high  needs  of  fast  disk  I/O 
in  order  to  display  time-dependent  data  for  the  post-processing.  In  addition,  the  multi-processor 
workstations  have  allowed  us  to  develop  and  test  our  Navier-Stokes  codes  locally  and  improve  their 
performance  before  performing  production  mns  at  the  DOD  High-Performance  Computing  Centers. 


Equipment  purchased  with  funding  from  DURIP  grant  F  49620-95-1-0123  : 

Silicon  Graphics  Workstations  $367,846 

Graphics  Workstation 

[Power  Onyx  Reality  Engine2  by  Silicon  Graphics  Inc.] 

Including:  Power  Onyx  Reality  Engine2  Deskside 

2  R8000/90  CPUs  w/  4MB  secondary  cache 

512  MB  Memory 

4  GB  System  Disk 

4  GB  Differential  Disk 

2  GB  DAT  Internal  Drive 

680  MB  CD-ROM  Drive 

FDDI  Controller 

21"  MultiSync  Monitor 

IRIX  5.3  System  Software 

FORTRAN  and  C  Compiler 

Program  Development  Option 

Network  File  System 

Installation 


19971203  265 


Two  Power  Challenge  Workstations 

[Power  Challenge  L  by  Silicon  Graphics  Inc.] 

Including:  Power  Challenge  L  Deskside 

4  R8000/90  CPUs  w/  4MB  secondary  cache 

512  MB  Memory 

4  GB  System  Disk 

4  GB  Differential  Disk 

IRIX  5.3  System  Software 

Network  File  System 

FDDI  Controller 

Installation 

Power  Indigo  Graphics  Workstation 

[Power  Indigo  Extreme  by  Silicon  Graphics  Inc.] 
including;  Power  Indigo  Extreme 

1  R8000/75  CPU  w/  2MB  secondary  cache 

128MB  Memory 

4  GB  System  Disk 

8  GB  Differential  Disk 

IRIX  5.3  System  Software 

Network  File  System 

FDDI  Controller 

Installation 

Three  Indy  Workstations 

[Indy  by  Silicon  Graphics  Inc.] 
including:  Indy 

1  R5000/150  CPU 
64MB  Memory 
1  GB  System  Disk 
IRIX  5.3  System  Software 
Network  File  System 
Installation 


Peripherals  and  Accessories 
Macintosh  Computer 

[PowerMac  Performa  and  printer  by  Apple  Computer,  MacWarehouse] 
Hard  Disk  Array 

[  36  GB  Hard  Disk  Array  Striped  for  Optimal  Performance 
by  Nordisk,  Four  Comers,  Falcon] 

Including;  Fast  Wide  Differential  SCSI-2  Host  Adapter 

9  Seagate  Barracuda  4  GB  SCSI  Drives 
Host  Controller  SCSI  Terminator 
6’  Wide  Differential  SCSI  Cable 

Accessories 

[Graybar,  UA  Stores,  Crescent  Electric,  MacWarehouse,  Laser  Experts] 

including:  FDDI  Cable 

Computer  Cable 
Printer  Cartridges 
Data  Tapes 

TOTAL  COST 

University  of  Arizona  Cost  Sharing 

DOD  COST 


$  7,642 


$375,488 
$  75,097 
$300,391 


Summaries  of  Research  Projects  for  which  Equipment  Was  Used 

DOD  Agency:  Air  Force  Office  of  Scientific  Research 
Contract  Number:  F49620-94- 1-0208  (Dr.  James  McMichael) 

Title:  Numerical  Simulations  of  Wall  Jets 
Principal  Investigator:  H.  Fasel 
Duration:  March  15, 1994-March  14, 1997 
Amount:  $189,790 

Wall  jets,  or  wall-jet-like  flows,  are  of  great  technical  relevance  for  numerous  aerospace  applications.  Wall 
jets  can  be  used  as  efficient  means  of  controlling  external  and  internal  boundary  layers.  For  external 
applications,  wall  jets  can  be  employed  advantageously  for  circulation  control  of  airfoils  and  thus  can  serve 
as  efficient  hit-augmentation  devices.  Such  hft  augmentation  can  be  accomphshed  by  either  tangential 
blowing  (through  the  wing  section)  along  the  upper  surface  of  airplane  wings  or  by  employing 
multi-segment  airfoils  (for  example,  slotted  flaps).  Wall  jets  are  also  highly  relevant  for  modem  jet 
engines,  where  wall  jets  occur,  for  example,  in  the  flow  over  a  cowl  of  a  fan-jet  engine  or,  internally,  in  the 
flow  over  film-cooled  turbine  blades. 

In  spite  of  the  great  technical  importance  of  wall-jet  flows,  relatively  httle  is  known  about  transitional  and 
turbulent  wall  jets.  This  is  due  to  the  complicated  physics  involving  the  evolution  and  intera.ction  of  large 
(coherent)  stmctures.  From  experimental  evidence,  it  is  known  that  the  large  stmctures  dominate  the  flow 
behavior.  In  particular,  the  behavior  of  the  structures  appears  to  be  controlled  by  nonlinear  effects,  which 
makes  theoretical  investigations  very  difficult.  Experiments  are  also  difficult  and  costly  to  perform. 
Therefore,  in  this  research  project.  Direct  Numeric^  Simulations  (DNS)  and  Large-Eddy  Simulations 
(LES)  are  employed,  with  the  goal  of  understanding  the  fundamental  mechanisms  that  power  the  dynamic 
behavior  of  the  large  stmctures.  With  that  knowledge,  chances  are  much  higher  of  finding  strategies  to 
favorably  control  these  flows,  namely,  by  manipulating  the  large  stmctures.  Toward  this  end,  numerical 
methods  for  solving  the  complete  Navier-Stokes  equations  for  incompressible  flows  have  been  developed 
that  are  tailored  for  efficient  simulations  of  wall  jets.  The  numerical  codes  developed  for  this  purpose 
employ  highly  accurate,  compact,  finite  difference  approximations  in  the  downstream  and  waU-normal 
directions  while  using  pseudospectral  approximations  iii  the  spanwise  direction.  It  is  important  to  note  that 
the  underlying  numerical  model  is  "spatial,"  thus  allowing  realistic  spatially  developing  flow  simulations. 
In  this  research  project  we  are  at  a  stage  where  preliminary  direct  simulations  are  being  carried  out  and, 
therefore,  the  enhanced  post-processing  capability  of  the  aquired  equipment  has  been  of  great  help.  The 
high-performance  workstations  have  been  absolutely  essential  for  tmly  scmtinizing  the  data  of 
thefull-scale  simulations.  The  codes  for  the  LES  are  still  in  development  and  the  pre-processing  support 
from  the  workstations  have  considerably  shortened  the  code  development  time. 


DOD  Agency:  Air  Force  Office  of  Scientific  Research 
Contract  Number:  F49620-94-1-0131  (Dr.  James  McMichael) 

Title:  Film  Cooling  by  a  Pulsating  Wall  Jet:  Experiments,  Numerical  Simulation,  and 
Stability  Theory 

Principal  Investigators:  I.  Wygnanski,  A.  Ortega,  and  H.  Fasel 
Duration:  January  15,  1994-January  14,  1997 
Amount:  $469,018 


Wan  jets  have  both  a  fundamental  and  practical  significance  in  convective  heat  transfer.  In  the  presence  of 
a  free  stream,  tangential  blowing  near  a  solid  surface,  a  wall  jet,  is  widely  used  for  film  cooling 
applications,  such  as  film-cooled  gas  turbine  blades,  combustion  chambers,  exhaust  nozzles  of  rockets,  etc. 
On  the  other  hand,  a  heated  wall  jet  can  be  used  to  prevent  icing  of  airplanes  or  to  de-ice  them.There  has 
been  considerable  interest  in  film-cooling  (-heating)  using  wall  jets.  However,  as  evidenced  by  the 
extensive  research  literature,  many  fundamental  issues  regarding  the  efficiency  of  wall  jets  are  not  yet 
understood.  We  do  know  that  the  large  (coherent)  structures  of  turbulent  wall  jets  play  a  decisive  role  in  the 
heat  transfer  mechanism,  and  thus  the  usefulness  of  wall  jets  for  film  cooling.  From  our  past  research  at 


The  University  of  Arizona,  we  also  know  that  the  evolutionary  and  dynamical  behavior  of  the  large 
stmctures  can  be  influenced  by  controlled,  time-dependent  forcing.  For  example,  for  a  "cold"  wall  jet 
blown  over  a  curved  surface,  periodic  pulsing  of  the  wall  jet  (even  with  every  small  amplitudes)  increases 
the  region  where  the  flow  stays  attached  to  the  wall  before  separation  sets  in.  In  addition,  due  to  nonhnear 
effects,  the  wall  jet  organizes  the  stractures  in  such  a  way  that  the  mean  skin  friction  is  reduced. 
Considering  the  effects  that  pulsing  has  on  the  dynamical  behavior  of  the  wall  jet,  the  effect  on  heat  transfer 
might  turn  out  to  be  even  more  striking  and  might  lead  to  new  knowledge  that  could  have  considerable 
impact  on  airplane  design  and  development. 

Because  of  the  considerable  importance  of  unsteady  pulsating  wall  jets,  an  extensive 
numerical-experimental  research  program  was  initiated  at  The  University  of  Arizona.  In  parallel  with 
experimental  investigations.  Direct  Numerical  Simulations  (DNS)  and  Large-Eddy  Simulations  (LES)  have 
been  carried  out  that  allowed  direct  comparison  of  numerical  and  experimental  results.  Toward  this  end, 
we  have  developed  numerical  methods  for  solving  the  complete  Navier-Stokes  equations  for  compressible 
flows,  although  at  first  only  incompressible  (low  subsonic)  applications  are  being  considered.  Using  the 
compressible  equations  allows  for  total  coupling  of  all  temperature  effects  with  fluid  dynamical  effects, 
such  as  temperature-dependent  density,  viscosity,  etc. 


DOD  Agency:  Army  Research  Office 

Contract  Number:  DAAL03-92-G-0228  (Dr.  Thomas  Doligalski) 

Title:  Numerical  Investigation  of  Transitional  and  Turbulent  Axisymmetric  Wakes  at 
Supersonic  Speeds 
Principal  Investigator:  H.  Fasel 
Duration:  May  1,  1992- April  30, 1995 
Amount:  $440,174 


A  comprehensive  effort  is  being  undertaken  to  investigate  transitional  and  turbulent  axisymmetric  wakes 
behind  cylindrical  bodies  aligned  with  the  flow  at  supersonic  speeds.  Particular  emphasis  is  on  identifying 
and  understanding  the  dynamical  behavior  of  the  large-scale  vortical  stmctures  that  control  the  flow 
behavior  in  a  supersonic  wake.  Direct  Numerical  Simulations  (DNS)  and  Large-Eddy  Simulations  (LES) 
are  the  main  investigative  tools.  The  numerical  simulations  are  supported  and  complemented  by  a 
theoretical  effort  based  on  stability  theory  analysis  and  Proper  OrAogonal  Decomposition  (POD) 
techniques,  which  are  applied  to  the  numerically  generated  data. 

For  axisymmetric  aerodynamic  bodies  in  supersonic  flight,  the  flow  field  in  the  wake  region  has 
considerable  effect  on  the  aerodynamic  drag.  Even  small  changes  in  the  flow  behavior  of  the  wake  may 
affect  the  performance  of  the  entire  flight  vehicle,  e.g.,  missiles,  rockets,  or  projectiles.  The  effect  on  the 
aerodynamic  drag  is  mainly  due  to  the  recirculation  region  that  develops  in  the  base  region  of  the  body  and 
thus  to  the  low  pressure  associated  with  the  recirculating  flow  ("base  drag").  Flight  tests  with  projectiles 
(U.S.  Army  549  projectile)  have  shown  that  the  base  drag  may  account  for  up  to  35%  of  the  total  drag.  This 
suggests  that  attempts  to  modify  the  near-wake  flow  such  that  the  base  pressure  would  increase  could  be 
highly  rewarding  with  respect  to  drag  reduction  and,  as  a  consequence,  with  regard  to  increasing  the 
performance  characteristics  of  flight  vehicles  or  projectiles. 

It  is  well  known  that  for  subsonic  (incompressible)  wakes,  the  dynamics  of  the  large  (coherent)  structures 
play  a  dominant  role  in  the  local  and  global  behavior  of  the  flow.  This  evidence  was  found  from  both 
experimental  investigations  and  numerical  simulations  (including  ours)  and  was  confirmed  by  theoretical 
studies.  For  supersonic  speeds,  on  the  other  hand,  very  little  is  yet  known  about  the  dynamical  behavior  of 
turbulent  flows.  This  is  true  for  supersonic  flows  in  general  and  for  axisymmetric  wakes  in  particular. 

Thus,  the  question  arises:  Do  large  stmctures  play  a  similarly  important  role  for  supersonic  separated  flows 
and  in  particular  for  supersonic  axisymmetric  w^es?  There  are  few  experimental  investigations  that  have 
focused  on  this  issue.  However,  when  looking  at  flow  visualization  pictures  of  supersonic  wake  flows, 
distinct  patterns  with  large-scale  stmctures  can  be  observed.  For  supersonic  axisymmetric  wakes,  the  mean 
flow  stmcture  of  the  near-wake  region  is  characterized  by  the  axisymmetric  shear  layer  originating  at  the 
sharp  comers  of  the  blunt  base. 

Supersonic  axisymmetric  wakes  are  extremely  difficult  to  investigate  experimentally.  Wind  tunnel 


interference  and  interference  from  model  support  strongly  affect  the  mean  flow  behavior,  which  may  be  an 
indication  that  this  behavior  might  be  caused  by  the  presence  of  large  coherent  structures.  Because  of  the 
experimental  difficulties,  numerical  simulations  represent  a  new  alternative  for  investigating  the 
complicated  unsteady  flow  phenomena  in  the  supersonic  wake. 

Direct  numerical  simulations  using  the  complete  Navier-Stokes  equations  are  restricted  to  somewhat 
low-to-moderate  Reynolds  numbers  because  of  the  rapidly  increasing  demands  on  computing  power  as  the 
Reynolds  number  increases.  Therefore  in  the  course  of  this  research,  we  have  been  working  on  ways  to 
drastically  increase  the  algorithmic  efficiency  of  our  Navier-Stokes  codes.  We  have  found  that  our  basic 
codes  require  relatively  little  overhead  costs  when  run  on  parallel  machines. 

Because  of  the  Reynolds  number  limitations  on  direct  simulations,  we  have  also  performed  Large-Eddy 
Simulations  (LES)  using  subgrid-scale  turbulence  models.  Because  our  Navier-Stokes  equations  are 
highly  suited  for  implementation  of  subgrid-scale  turbulence  models,  we  are  able  to  use  essentially 
identical  codes  for  both  the  direct  and  large-eddy  simulations.  This  allows  proper  fine-tuning  of  the  LES 
codes  (in  particular,  of  the  subgrid-scale  models)  so  that  they  can  be  applied  with  greater  confidence  to  the 
higher  Reynolds  number  calculations. 


DOD  Agency:  Army  Research  Office 

Contract  Number:  DAAH04-93-G-0393  (Dr.  Thomas  Doligalski) 

Title:  Investigation  of  Coherent  Structures  in  the  Shear  Layers  of  Supersonic  Wakes 

Principal  Investigator:  H.  Easel 

Duration:  September  1,  1993- August  31,  1996 

Amount:  $83,000 


This  research  is  funded  by  the  AASERT  Program.  The  goal  of  these  investigations  is  to  contribute  toward 
an  understanding  of  the  fundamental  mechanisms  that  are  responsible  for  the  evolution  and  dynamical 
behavior  of  the  coherent  structures  in  the  shear  layers  of  a  supersonic  wake.  The  detailed  investigation  of 
the  dynamical  behavior  of  the  shear  layers  synergistically  enhances  the  main  study  of  the  global  dynamical 
behavior  of  supersonic  axisymraetric  wakes  which  is  being  carried  out  through  funding  from  the  parent 
grant,  DAAL03-92-G-0228  (as  summarized  above). 

For  these  investigations,  we  also  employed  Direct  Numerical  Simulations  (DNS)  and  Large-Eddy 
Simulations  (LES).  Numerical  methods  and  computer  codes  for  such  simulations  were  developed  with 
funding  from  the  parent  grant.  The  numerical  simulations  were  supported  by  theoretical  investigations 
using  both  the  classical  stability  theory  and  Parabolized  Stability  Equation  (PSE)  approach.  In  addition  to 
investigations  of  axisymmetric  shear  layers,  a  major  focus  was  placed  on  shear  layers  in  a  plane  wake.  This 
enabled  detailed  and  rigorous  comparison  of  the  simulations  to  measurements  from  an  experimental  effort 
(for  a  plane  shear  layer)  being  conducted  at  The  University  of  Illinois. 


DOD  Agency:  Office  of  Naval  Research 

Contract  Number:  NOOOl 4-94- 1-0095  (Dr.  Patrick  Purtell) 

Title:  Effects  of  Adverse  Pressure  Gradient  and  Wall  Curvature  on  the  Turbulence 
Mechanisms  in  Boundary  Layer  Flows:  Reynolds-Averaged  Navier-Stokes 
Calculations,  Large-Eddy  Simulations,  and  Direct  Numerical  Simulations  for  the 
Stratford  Ramp 

Principal  Investigator:  H.  Easel 

Duration:  October  1,  1993-September  30,  1996 

Amount:  $417,653 

The  main  goal  of  this  research  project  is  to  identify  and  understand  the  combined  effects  of  strong  adverse 
pressure  gradient  and  strong  streamwise  curvature  on  the  turbulence  mechanisms  of  turbulent  boundary 
layers  at  high  Reynolds  numbers.  For  these  investigations,  the  so-called  Stratford  ramp  was  chosen  as  a 
generic  flow  geometry.  For  this  geometry,  both  adverse  pressure  gradient  and  streamwise  curvature  act  on 
die  turbulent  boundary  layer  in  a  unique  way:  the  boundary  layer  is  continuously  on  the  verge  of 
separation.  This  flow  geometry  was  highly  suited  for  our  attempts  to  identify  and  understand  the  relevant 
turbulence  mechanisms  and,  as  a  consequence,  for  developing  better  turbulence  models  for  nonequihbrium 


turbulent  boundary  layers.  For  these  investigations,  Reynolds-Averaged  Navier-Stokes  Calculations 
(RANS),  Large-Eddy  Simulations  (LES),  and  Direct  Numerical  Simulations  (DNS)  were  employed. 
Special  focus  was  on  performing  computations  for  Reynolds  number  regimes  where  either  LES  and  RANS 
or  DNS  and  LES  can  be  carried  out  simultaneously.  The  Direct  Numerical  Simulations  are  crucial  to  the 
LES  effort,  namely  for  developing  improved  subgrid-scale  models.  Post-processing  numerical  data  [using 
diagnostic  tools  such  as  interactive  time-dependent  graphics,  linear/nonlinear  stability  theory,  parabolized 
stability  equations  (PSE),  proper  orthogon^  decomposition  (POD),  etc.]  obtained  from  the  overlapping 
calculations  will  greatly  enhance  chances  for  identifying  the  relevant  mechanisms,  which  then  can  be 
incorporated  in  improved  turbulence  models.  In  order  to  efficiently  perform  these  post-processing  tasks  for 
the  massive  amount  of  time-dependent  data  generated  by  Ae  simulations,  the  high-performance 
workstations  have  been  vital  to  this  project.  The  computational  effort  was  conducted  in  direct  and  close 
collaboration  with  an  experimental  effort  by  I.  Wygnanski  (The  University  of  Arizona)  and  a  theoretical 
turbulence  modeling  effort  by  C.  Speziale  (Boston  University).  In  addition,  this  group  effort  is  linked  to  the 
computational  effort  by  T.  Huang  (David  Taylor  Model  Basin),  who  will  test  the  improved  Reynolds  stress 
models  that  result  from  our  investigations  for  Navy  relevant  geometries. 


Summary  of  How  the  Aquired  Instrumentation  Has  Enhanced  the  DOD-Funded  Research 

The  instmmentation  has  significantly  supported  our  DOD-funded  research  and,  as  a  consequence,  has 
enabled  major  progress  in  DOD-relevant  Computational  Fluid  Dynamics  research  that  would  otherwise  not 
have  been  possible.  In  particular,  with  the  high-speed  workstations  aquired,  we  were  able  to  close  an 
existing  gap  that  had  prevented  us  from  fuUy  utilizing  the  massive  computing  power  available  to  us  at  the 
DOD  High-Performance  Computing  Centers  and  the  NSF  Supercomputer  Centers.  As  summarized  above, 
we  were  engaged  in  several  DOD-sponsored  research  projects  requiring  highly  challenging  numerical 
simulations  of  complex  flows  that  have  never  been  attempted  before.  We  were  numerically  simulating 
transitional  and  turbulent  flow  phenomena  in  a  variety  of  flow  geometries,  such  as  wall  jets  (with  and 
without  heat  transfer;  funded  by  AFOSR),  supersonic  wakes  (supported  by  ARO),  and  boundary  layers 
along  flat  and  curved  walls  (supported  by  ONR).  In  all  cases,  the  flows  considered  were  strongly  time 
dependent  and  the  phenomena  to  be  investigated  are  highly  nonlinear.  Flow  speeds  were  from  low 
(incompressible)  to  high  (supersonic). 

In  all  the  projects.  Direct  Numerical  Simulations  (DNS)  and  Large-Eddy  Simulations  (LES)  were  carried 
out  with  the  goal  of  pushing  the  relevant  Reynolds  number  to  as  high  a  value  as  currently  feasible  with 
available  supercomputers.  The  DNS  are  based  on  solving  the  complete  Navier-Stokes  equations  with  no 
further  assumptions  other  than  those  made  for  deriving  Aese  equations.  The  LES  require  subgrid-scale 
models.  For  LES,  the  grid  spacing  is  such  that  only  the  larger  scales  are  resolved  directly  by  the 
computational  grid.  Scales  smaller  than  the  grid  have  to  be  modeled. 

In  all  the  projects,  the  underlying  computational  model  was  "spatial."  Thus,  the  structures  evolving  in  the 
transitional  and  turbulent  flows  could  develop  (grow  or  decay)  not  only  in  time,  but  also  in  the  spatial 
(downstream)  direction,  as  is  the  case  in  realistic  laboratory  experiments  or  in  free  flight.  This  spatial 
model  is  in  contrast  to  the  so-called  "temporal"  model,  for  which  spatial  (downstream)  periodicity  is 
assumed  and  which  therefore  introduces  assumptions  that  are  not  realistic  for  many  situations.  The 
temporal  model  allows  for  short  computational  domains  and  thus  considerably  reduces  the  computational 
demands  and,  in  particular,  the  size  of  the  data  files  produced  by  the  numerical  simulations. 

The  realistic  spatial  simulation  model,  on  the  other  hand,  lead  to  considerably  higher  computational 
complexity  because  of  the  typically  much  larger  (in  the  downstream  direction)  computational  domains 
required  and  because  of  the  difficulties  associated  with  providing  and  implementing  proper  inflow  and 
outflow  boundary  conditions.  As  a  consequence  of  employing  the  spatial  model,  considerably  more  time 
and  effort  were  necessary  for  code  development.  The  numerical  methods  had  to  be  very  accurate  and 
efficient  for  simulations  with  such  large  grid  sizes  to  be  at  all  feasible  on  supercomputers.  These  efficiency 
and  accuracy  requirements,  together  with  the  difficulties  associated  with  inflow  and  outflow  conditions, 
required  extensive  testing  in  the  development  phase  of  the  computer  code.  For  this  reason,  adequate 
pre-processing  capabilities  has  been  essential  for  effective  utilization  of  available  supercomputing 
resources.  The  high-performance  workstations  acquired  (Power  Challenges  in  connection  with  the  Indy’s) 
have  met  our  pre-processing  needs.  The  pre-processing  capability  of  the  workstations  has  enabled  us  to  do 
all  of  our  pre-processing  locally.  This  has  helped  to  relieve  the  network  and  to  unclog  the  remote 


supercomputers.  After  all,  the  supercomputers  are  best  utilized  for  number  crunching,  and  using  it  for  code 
development  purposes,  which  can  be  done  more  efficiently  on  considerably  less  expensive  machines,  is  in 
fact  a  waste  of  supercomputing  resources. 

In  addition,  post-processing  of  the  massive  amount  of  data  that  results  from  our  simulations  had  been  very 
cumbersome  (if  possible  at  all)  when  carried  out  remotely  at  the  supercomputing  centers.  Even  attempting  it 
had  been  wasteful  of  supercomputing  resources.  Our  post-processing  needs  have  been  met  more 
efficiently  and  cost  effectively  on  the  powerful  high-performance  workstations.  The  flows  that  we  were 
studying  were  dominated  by  the  generation  and  evolution  of  coherent  turbulent  structures.  These 
three-dimensional,  non-periodic  structures  cover  a  wide  range  of  spatial  and  temporal  scales.  Due  to  their 
non-periodic  nature,  their  dynamics  cannot  be  analyzed  by  means  of  Fourier  transforms.  Therefore, 
detailed  investigations  of  the  space-time  evolution  of  the  physical  flow  field  was  essential  to  gaining  an 
understanding  of  the  dynamics  and  the  underlying  physics  of  the  turbulent  structures.  This  had  important 
implications  for  the  post-processing  of  the  data  generated  in  our  numerical  simulations. 

Since  it  is  the  rapid  changes  of  the  flow  field  with  time  that  was  important,  the  most  revealing  information 
concerning  the  dynamical  behavior  of  the  flow  structures  was  obtained  by  scrutinizing  animations  of  the 
unsteady  data,  rather  than  snapshots  of  the  flow  depicting  its  state  at  one  instant  in  time.  In  order  to 
generate  these  animations  at  every  time  step,  the  data  first  had  to  be  read  from  disk  into  memory.  Then  the 
data  were  processed  to  extract  useful  information,  e.g.,  to  compute  level  surfaces  of  flow  variables,  particle 
traces,  kinetic  energy  levels,  entropy  distributions,  etc.  Finally,  the  processed  data  were  sent  to  the  graphics 
processor  for  display  on  the  screen.  This  whole  process  was  then  repeated  for  the  next  time  step.  Typical 
data  sets  required  approximately  50  MB  of  disk  space  (and  50  MB  of  memory)  per  time  step,  where  a 
meaningful  animation  of  a  non-periodic  flow  required  several  hundred  time  steps.  These  post-processing 
demands  are  in  contrast  to  "mainstream"  CFD  applications,  where  only  a  steady-state  mean  flow  is 
computed  and  visualized. 

For  truly  interactive  time-dependent  graphics  processing,  the  powerful  graphics  workstations  from  Sihcon 
Graphics  (Power  Onyx  and  Power  Indigo)  in  combination  with  the  36-GB  hard  drive  array  with 
high-speed  disk  I/O  capable  of  simultaneously  reading  from  multiple  individual  disks  have  proven  to  be 
vital.  The  acquired  equipment  has  allowed  us  to  cut  the  time  required  to  generate  a  time-dependent 
animation  from  a  day  down  to  several  minutes.  The  gains  in  speed  resulted  from  two  factors:  (1)  The  use 
of  disk  array  (redundant  arrays  of  independent  disks)  technology  has  increased  the  data  transfer  rate  from 
the  disk  by  a  factor  of  up  to  10.  (2)  The  use  of  multiple  processors  has  increased  the  rate  of  data  processing 
once  the  data  are  in  memory.  An  advantage  of  the  acquired  Onyx  system  is  that  additional  processors  can 
be  added  as  processing  demands  increase  with  increased  computing  powers  of  new  supercomputers.  The 
resulting  speedup  has  resulted  in  a  qualitative  change  in  our  post-processing:  The  shorter  turn-around  time 
has  enabled  us  to  do  truly  interactive  graphics  processing. 


